1. Field of the Invention
The present invention relates to a multimedia broadcast service. More particularly, the present invention relates to a method and an apparatus for synchronizing media data and auxiliary data, which are received by a receiver from a broadcast network or a communication network, with each other in a heterogeneous network environment.
2. Description of the Related Art
Generally, in a multimedia service, audio, or video accompanied by audio (hereinafter, referred to as ‘regular media’), is provided and in some cases, when needed, auxiliary data (hereinafter, referred to as ‘adjunct media’), such as web documents, widgets, interactive applications, audio clips, video clips, graphics, texts, images, and the like, is provided together with the regular media. Some adjunct media may be provided temporally irrespective of regular media (hereinafter, referred to as ‘unscheduled adjunct media’), and when the adjunct media needs to be associated with the content of the regular media, the adjunct media may be provided to be played temporally in synchronization with a related part of the content of the regular media (hereinafter, referred to as ‘scheduled adjunct media’). In the latter, playing the adjunct media in synchronization with the related part of the content of the regular media is referred to as “media synchronization”.
The unscheduled adjunct media does not need to be synchronized with the content of the regular media, such that the receiver, upon receiving the unscheduled adjunct media, immediately plays the received unscheduled adjunct media. On the other hand, the scheduled adjunct media starts being played in a part related thereto in the content of the regular media, and in spite of some variations depending on characteristics of the adjunct media, the play of the scheduled adjunct media is terminated at the request of a user, upon occurrence of a channel change, or upon exhaustion of the adjunct media data.
The Digital Video Broadcasting (DVB) standard of the related art, European Telecommunication Standard Institute (ETSI) Telecommunications Standard (TS) 102 823 V1.1.1 (2005-11) defines data similar to the adjunct media as auxiliary data. In this specification, according to the Moving Picture Experts Group (MPEG) Media Transport (MMT) standards (International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Joint Technical Committee 1 (JTC1)/SubCommittee 29 (SC29)/Working Group 11 (WG11)), the concept of auxiliary data is extended to be suitable for smart TeleVisions (TVs) in a heterogeneous network environment in which data is received using both a broadcast network and the Internet. In the MMT standards, the term “asset” is used as a concept similar to a stream. In this specification, a partial media stream indicating a partial media compression data stream is added to web documents, widgets, interactive applications, audio clips, video clips, graphics, texts, images, and the like, of a broadcast standard of the related art, and they will be defined as adjunct assets. A compression data stream of the regular media is defined as a regular asset.
FIG. 1 illustrates a partial media stream according to the related art.
Referring to FIG. 1, stereoscopic video includes left-eye view and right-eye view images, among which left-eye images 111, 113, and 115 are transmitted as a regular asset 110 and a right-eye image 121 is transmitted as an adjunct asset 120 only for a predefined period (e.g., for 1 minute). In this case, a receiver previously downloads the adjunct asset 120 over the Internet, and plays the adjunct asset 120 in frame-wise synchronization with the regular asset 110 when receiving the regular asset 110 in real time over a broadcast network, thereby presenting a stereoscopic video service in a time period of a predefined duration.
The MPEG-2 Digital Storage Media-Command and Control (DSM-CC) standard of the related art (ISO/IEC 13818-6:1998) or Digital Video Broadcasting (DVB) standard (ETSI TS 102 823 V1.1.1) separately configures and uses a timeline per program item, instead of a media synchronization clock, a System Time Clock (STC) of the MPEG-2, to synchronize content metadata or auxiliary data with audio or video. This is because, due to the possible existence of a non-contiguous point in a Program Clock Reference (PCR) based on the STC, the PCR cannot be used to configure a contiguous timeline which increases at a fixed rate from start to end of a program item. The timeline per program item is called as a metadata content timeline in the MPEG-2 DSM-CC standard, and is referred to as a DVB broadcast timeline in the DVB standard. Although the timelines in the two standards are different in terms of configuration scheme and name, they both are timelines for synchronization between regular media and adjunct media (hereinafter, referred to as a ‘media timeline’). Based on a media timeline of the related art, metadata is associated with a video segment or auxiliary data is executed in synchronization with video.
FIGS. 2A through 2D illustrate synchronization of an application according to a metadata content timeline in an MPEG-2 DSM-CC standard according to the related art.
Referring to FIGS. 2A through 2D, the application is an adjunct asset and is being synchronized with video, which is a regular asset. FIG. 2C illustrates a metadata content timeline for video of a regular program. This timeline is a continuous timeline over a play time from start to end of a video, but if due to a need for edition in a studio, the edition is made such that a new scene starts from a minute-based time, a discontinuous period of 1 minute or less may be included. FIG. 2D illustrates a metadata content timeline of an advertisement to be inserted between programs shown in FIG. 2C. In FIGS. 2A through 2D, there is an application, which is an adjunct asset to be executed (or run) at a particular point in time during the advertisement as intended by an advertisement creator. FIG. 2B illustrates a timeline according to an STC of the MPEG-2 system standard in a case of transmission of FIGS. 2C and 2D over a broadcast network. On an STC-based timeline, a discontinuous period may exist due to stream edition, stream splicing, or the like. To designate a particular time point on the STC-based timeline and execute an application in the designated time point, an STC at a corresponding time point, which changes every transmission of the same advertisement, has to be newly read and transmitted as a Presentation TimeStamp (PTS) of the application. In the MPEG-2 DSM-CC standard, to avoid this inconvenience, transmission is made as shown in FIG. 2A, but a time point at which the application has to be executed is designated based on a metadata content timeline B. In this case, a time with a relative time difference from an advertisement start point is designated as the time point for execution of the application, such that the execution time point is fixed without changing every transmission of the advertisement. However, for such synchronization, the receiver has to reconfigure, maintain, and manage a metadata content timeline A and the metadata content timeline B.
In order for the receiver to configure the media timeline as shown in FIGS. 2A through 2D, a server has to transmit control information for media timeline configuration as a separate stream. This process needs a separate TS packet stream, and a separate Packet Identifier (PID) has to be assigned to this stream. Moreover, the server needs to periodically transmit various descriptors through the separate stream, and the receiver needs to configure the media timeline by reading and handling the descriptors. In addition, for synchronization between a regular media and an adjunct media, a value on a media timeline regarding an adjunct media start point has to be delivered in a timestamp attached to the adjunct media, and the receiver has to convert this timestamp into a value on the STC, thus achieving media play synchronization.
FIG. 3 illustrates an application synchronization according to a DVB broadcasting timeline in a DVB standard (ETSI TS 102 823 V1.1.1) according to the related art.
To deliver the DVB broadcasting timeline to a receiver, the server incorporates a PTS into a header to deliver broadcast_timeline_descriptor through a Packetized Elementary Stream (PES) packet, and periodically transmits information of a pair (DVB broadcasting timeline value, PTS value). A timeline of each program segment is determined based on the DVB broadcasting timeline.
Referring to FIG. 3, a time of each program segment does not necessarily start from 0 and DVB broadcasting timelines of a regular program and an advertisement are continuous to each other. This is intended to prevent a reception side from detecting a discontinuous point on the DVB broadcasting timelines and deleting only the advertisement. An STC value corresponding to a point (3), a time point at which an application is to be executed, may be estimated by extrapolation using information of two pairs (i.e., a DVB broadcasting timeline value and a PTS value) transmitted immediately before the point (3), that is, points (1) and (2). Moreover, the reception side has to configure, maintain, and manage a media timeline, and thus a transmission side has to transmit control information for the media timeline and a receiver has to periodically read and process the control information.
Therefore, a need exists for a synchronization method and an apparatus capable of achieving synchronization between regular media and adjunct media without configuring a media timeline.
The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with regard to the present invention.